DOCUMENT RESUME 



ED 372 091 



TM 021 753 



AUTHOR 
TITLE 

SPONS AGENCY 
PUB DATE 
CONTRACT 
NOTE 



PUB TYPE 



James, Carolyn M« ; And Others 

The Role of Planning in Simple Digital Circuit 
Des ign. 

Office of Naval Research, Arlington, Va. 
Apr 94 

N00014-91-J-1680 

16p. ; Paper presented at the Annual Meeting of the 
American Educational Research Association (New 
Orleans, LA, Apr 1 4-8, 1994). 
Reports - Research/Technical (143) — 
Speeches/Conference Papers (150) 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



MF01/PC01 Plus Postage. 

Cognitive Processes; Concept Formation; ^Design; 
Designers; Graduate Students; Higher Education; 
^Planning; *Problem Solving; Protocol Analysis; 
Undergraduate Students 

^Digital Circuits; Expertise; Expert Novice Problem 
Solving 



ABSTRACT 

The role of planning in the design of digital 
circuits was studied in two experiments. One compared the 
plann ing-relat ed activities of expert designers and beginning 
students of design, and the other looked at the effect on 
circuit-design processes of forcing students to construct a global 
plan. In experiment 1, five undergraduate students (novices) and two 
advanced graduate students with outside experience (experts) designed 
a one-bit full subtracter, taking several design constraints into 
account. Subject think-aloud protocols were videotaped and analyzed. 
Only one of the experts formulated a global plan before beginning the 
problem; undergraduate students did not formulate global plans but 
did exhibit local planning* More cross-component planning was 
conducted by students who were deemed more skillful by instructors. 
In the second experiment, 20 undergraduate and graduate students 
performed the same problem and a job-skill problem. Analyses suggest 
that at the group level there is no effect of planning on errors nor 
on attempts at optimization. Forcing formulation of a global plan 
does not mean that subjects are able to carry it out. (Contains 13 
references.) (SLD) 
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Objectives 

One characteristic of expertise in digital circuit design is understanding 
how each part of the design interacts and interrelates to other parts. 
Instructors in introductory design courses often comment on how difficult it 
is to get students to think about their designs from this perspective. In other 
domains, planning often functions to provide problem solvers with a global 
view of their solution activity. This research investigates the role of planning 
in the design of digital circuits. In an initial experiment, the planning-related- 
activity of expert designers and beginning students of design was compared 
and a lack of global planning was observed. The objective of a second 
experiment was to look at the effect on circuit design processes of forcing 
students to construct a global plan. 

Theoretical Framework and Overview 
The knowledge and problem-solving processes that support the 
execution of complex cognitive skills and performances have been the subject 
of cognitive research for several decades. A focus of this research has been on 
characterizing differences between individuals who are experts and those 
who are novices. Such contrastive analyses have been undertaken, in part, to 
elucidate the "end state" performance goals of acquiring cognitive skills and, 
in part, to understand how to assist individuals in attaining these end states. 
Researchers have taken this approach in a number of domains including 
physics, chess, architecture, mechanical engineering and instructional design 
(e.g., Akin, 1988; Chase & Simon, 1973; Chi, Feltovich, & Glaser, 1981; Foz, 
1973; and Goel & Pirolli, 1989). 
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Characteristics of Expertise 

Previous research has resulted in some general characterizations of the 
attributes of experts and novices in terms of both overall approach and 
specific strategies (e.g., Ericsson & Smith, 1991; Chi, Glaser, & Farr, 1988; 
Larkin, 1980). Several differences between expert and novices are noteworthy. 
For example, Chi, Feltovich and Glasei (1981) foand that experts represent 
problems in their domain at a deeper level than novices; and Paige and 
Simon (1966) showed that experts spent more time analyzing a problem 
qualitatively. In addition, Larkin (1980) has shown that an expert first plans 
the main features of the solution and then elaborates them into the complete 
solution rather than initially delving into the detailed solution process. 

There have been a number of studies of the design process in 
architecture and in various areas of engineering (e.g., Akin, 1988; Foz, 1973; 
Goel & Pirolli, 1989; Vandermolen, James, Goldman, Biswas, & Bhuva, 
1992). Although design appears to be a highly constructive and possibly 
idiosyncratic process, there are a number of general principles and heuristics 
that students of design are taught (e.g., Garrod & Boms, 1991). As in other 
areas of complex cognitive activity, researchers have attempted to characterize 
the differences between individuals who have acquired different levels of 
design skill. 

Results in the design domain generally support results found by other 
researchers. Glaser and Chi (1988) noted that experts spend more time 
analyzing a problem qualitatively, have strong self monitoring skills, and 
solve problems faster. Foz (1973) compared novice and expert designers and 
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concluded that experts employ a breadth-first approach to exploring the 
design space, do not commit to a solution until alternatives are explored, and 
use solutions from prior experience to solve new problems. One interesting 
observation is the fact that experts even consider solutions that have no a 
priori likelihood to succeed. 

The present research investigates the role of planning in designing a 
complex domain, digital circuit design. Two experiments are reported. The 
first experiment examined the spontaneous use of planning during the 
design of a combinational logic circuit. The conclusions from this study are 
that students did not spontaneously plan before they started solving a design 
problem. But students did exhibit a great deal of "local" planning, i.e., 
planning that occurred during the solution process at times when students 
got confused or needed to stop and figure out what to do next. 

The results of Experiment 1 with respect to planning led us to conduct 
Experiment 2 in which students were forced to generate a global plan before 
they started to solve a circuit design problem. The purpose of these forced 
global plans is to have the students think through the solution before 
beginning. The hypothesis was that such forced planning would foster more 
expert-like behavior and potentially better solutions. 

Experiment 1 

The purpose of the Experiment 1 was to extend the characterization of 
expertise to the area of circuit design. This is an interesting domain because 
simple design tasks can be executed as a set of standard procedures; however, 
to achieve proficiency even in simple design tasks, one has to be aware of 
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trade-offs between different performance parameters. Typical parameters 
include total area occupied, input-output delay, and total power 
consumption. As a result, a number of alternative solutions may be possible, 
each of which satisfies the functional requirements, but only one of which is 
optimal considering all requirements. Furthermore, most design problems 
can be decomposed into subproblems that can be solved independently. Doing 
so, however, ignores the interaction between the parts and often leads to 
suboptimal solutions. 

Experts and students enrolled in their first course in digital design 
solved a relatively simple combinational logic circuit problem. Think-aloud 
protocols were taken and provided the basis of analyses of planning that 
occurred when they were designing the circuit. 
Method 

Participants . Five undergraduate student volunteers participated in 
the study. All five students were currently enrolled in a senior-level digital 
design course. The students had all completed one introductory course in 
digital circuits, and had very limited design experience. Contrary to much 
previous research on expertise, the experts were not college professors or 
instructors in design. The experts consisted of two advanced graduate 
students in electrical engineering who had experience working outside of the 
University setting. 

Procedure. The problem required the subjects to design a one-bit full 
subtracter, taking into account several design constraints. The problem was 
similar to an example often used in class (the adder circuit), but different 
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enough so that neither novices nor experts could generate the solution from 
memory. The circuit required three inputs and two outputs, and the specified 
target representation was a circuit at the gate level. The subjects were 
provided with a written, functional description of the problem, rather than 
the desired input-output behavior. They were asked to think aloud during 
the solution process, which was videotaped. Each subject solved the problem 
independently and without a time constraint. Immediately following the 
problem solving session each subject was asked to watch the tape of her/his 
session and to add additional comments of what they were thinking and why. 
This retrospective session was also videotaped. 
Analysis and Results 

The protocol videotapes for the problem were analyzed at two levels: 
standard components of solution and discrete behavioral episodes. The 
component analysis reflects the standard sequence of solution components 
that is taught for doing combinational circuit design: understand problem, 
generate Truth table, generate K-map, generate Boolean expression, 
implement, and evaluate. Behavioral episodes were related to these standard 
design components. Episodes characterize the nature of the activity the 
subject was performing at a particular point in the solution. Five types of 
episodes (out of a total of 12 types) are associated with Planning-Related 
Activity; these are evaluate, list, plan, select and verify. The five Planning- 
Related Activity episodes are related to decision processes and are associated 
with the direction the problem solving takes (James, Goldman, and 
Vandermolen, 1994). 
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Highlights of the results of these analyses are the following: 

1. Only one subject, the expert EM, formulated an overall plan which 
outlined his entire solution, before he began the problem. 

2. In general, the students in this study did not formulate a global plan 
but they did exhibit local planning throughout their solution. 

3. The less skilled student subjects had more occurrences of the 
Planning-Related Activities within the Components, suggesting that they 
were having problems with the mechanics of the pr :edures. 

4. Students who were rated by their course-instructor as more skilled 
designers tended to have a greater degree of cross-component Planning- 
Related Activity than did those who were rated as less skilled. (These rating 
were independent of performance in Experiment 1.) This suggests that they 
were evaluating their solution and planning the next step. 

Summary and Rationale for Experiment 2 

Students who engaged in a great deal of within-component planning 
appeared to have little idea of the purpose of the design components they 
were attempting to execute. Even the more skilled students seemed to have 
some trouble determining what they should do at the conclusion of particular 
components. Accordingly, we decided to examine whether being forced to 
explicitly plan out an entire solution would lead to better performance on the 
design process. 

Experiment 2 

Students solved two combinational logic circuit design problems. On 
the first problem we observed the incidence of spontaneous planning-related 
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activity. On their second problem, students were specifically required to 
produce a global solution plan prior to beginning to work the problem. 
Think-aloud protocols were taken during solution of both problems. 
Method 

Participants . Twenty undergraduate and graduate student volunteers 
participated in the study. The students had all completed one introductory 
course in digital circuits and had had very limited design experience. The 
students were paid for their participation. 

Procedure. The subjects were divided into two groups. Each group 
was matched on performance in their current design course and on gender. 
Each subject was asked to solve two problems, one per experimental session. 
Before solving the second problem each subject was asked to formulate an 
overall plan for solving the problem. 

Problem A, the subtracter problem, was the same problem from the 
first experiment. Problem B, the job skill problem, required the subjects to 
solve a problem in which the input/output relations were described in words 
rather than symbols. It is the same type of problem as Problem A - a 
combinational logic problem. The circuit from Problem B required four 
inputs and two outputs. The specified target representation for both problems 
was a circuit at the gate level. Counterbalancing techniques assured that each 
problem served as the "first" problem an equal number of times across 
subjects. 

The subjects solved the problems using a computer simulation tool. 
They were asked to think aloud during the solution process which was 
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videotaped. Each subject solved the problems independently and without a 
time constraint. After solving the second problem, each subject was asked a 
series of questions that compared the two problems, probed for problem 
solving strategies, and elicited a discussion of how useful the subject felt it 
was to formulate the global plan before solving the second problem. 
Analysis and Results 

Three methods of analyses were employed: a component analysis, an 
accuracy analysis, and an optimization analysis. 

Component Analysis . The component analysis breaks down the design 
problem into domain-specific activities like 'generate truth table' and 
'generate Karnaugh map 1 . It could be argued that not all of the subjects knew 
the standard design algorithm and thus did not include the same number of 
components in their designs. An ANOVA was performed with group 
(solved problem B first, solved problem A first) as the between subjects factor, 
planning (no plan, plan) as the within subjects factor and total number of 
components as the dependent variable. None of the main effects nor the 
interactions were significant. Mean number of components was 9.45 for the 
no plan condition and 8.9 for the plan condition, out of a possible ten 
components. This result is consistent with our initial assumption that the 
subjects did know the standard algorithm for solving this class of problems. 
Each group included the same basic set of steps in solving the problems. 

Accuracy Analyses . The accuracy analysis investigates the presence of 
errors with respect to the planning condition. An ANOVA was performed 
with group (solved problem B first, solved problem A first) as the between 
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subjects factor, planning (no plan, plan) as the within subjects factor and total 
number of correct outputs as the dependent variable. There were no 
significant main effects nor interactions: mean numbers of correct outputs 
was .9 and 1.0, with a maximum of two outputs. 

The number of errors in the problem solutions were submitted to an 
ANOVA in which group (solved problem B first, solved problem A first) was 
the between subjects factor, planning (no plan, plan) was the within subjects 
factor, and total number errors in each solution was the dependent variable. 
Again, there were no significant effects. The mean number of errors per 
solution was 2.7 for the no plan and 4.6 for the plan conditions. 

Optimization Analysis . The optimization analysis examines whether 
planning led to increases in the attempt to optimize the design. Altogether 
there were only seven subjects who attempted to optimize any of the outputs 
in either plan condition. In all cases where a subject attempted to optimize a 
particular solution, both outputs were attempted. There were no 
circumstances in which a subject tried to optimize one output in a problem 
and ignored the other. There was only one subject who did not try to 
optimize in the no plan condition but who did try to optimize in the plan 
condition. Two subjects attempted to optimize in both the plan and the no 
plan condition and the remaining four subjects only attempted to optimize in 
the no plan condition. Thus, being forced to construct a global plan did not 
generally lead to a greater number of attempts to optimize a solution. 

Subject Interviews . In the post-design interviews the subjects were all 
asked if they felt that formulating the global plan was helped: 60% felt that 
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the global plans helped and 40% felt that it was not helpful. Of the 60% who 
felt the planning was helpful, only 16.6% (2 subjects) were able to correctly 
design more than two of the four outputs required for the entire study. 
Whereas, of the 40% who did not find planning helpful, 50% (4 subjects) were 
able to correctly design more than two of the four outputs required for the 
entire study. 

Summary . These analyses suggest that at the group level there is not 
an effect of planning on errors nor attempts at optimization. There are 
several possibilities for why the global plans were not associated with more 
successful designs: 

1. The plans were ignored or not followed during the actual design. A 
plan that was ignored obviously would not have an effect on the solution 
process, 

2. The plans were not correct, or were insufficiently detailed. An 
incorrect plan could lead to more rather than fewer errors and an 
insufficiently detailed plan would leave the subject either to replan or 
flounder. 

One of the more successful subjects offered his own suggestion as to 
why the forced global plan was not helpful: 

It really wasn't too helpful for formulating a global problem, a global 
plan, because I feel a global plan comes in when the design is much 
bigger. See if I have this design, and suppose I'm designing something 
big, a computer of this sort, then I definitely need a global plan. I can 
say that I can't just go designing bits and pieces. It would be useless. ... 
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OK, let me put it this way. The harder. There are two criterions. One 
is the hardness of the problem and the other is the volume of the 
design. 

Perhaps the planning was not evident because the better designers 
automatically plan, as shown in this statement from such a subject: "Well, it 
[planning] didn't help much because I did the same thing on the first one 
[problem] only I just did not explicitly state it" There is also the possibility 
that global planning is not useful to less skilled designers, or is only useful at 
certain junctures in their design problem solving. Forcing someone with 
insufficient knowledge and skills to formulate a plan does not mean that they 
are able to adequately flesh out the details to implement the plan. This could 
contribute to circumstances in which the designer knows the design 
algorithm but not necessarily how to fully implement each step. 

Further Analyses . Further analyses are underway, including an 
analysis of the protocols of the forced global plans. These analyses include a 
characterization of each statement in the protocols of the plans and an 
investigation of individual differences with respect to performance and 
attempts at optimization. 

The forced global planning statements generated by the subjects have 
been divided into two categories: Planning Statements and Other Statements. 
The Planning statements include: Action, Evaluative, Explanative, and 
Problem-Constraint Statements. The Other Statements include: Elaborative, 
Irrelevant, Repetitious, and Rereading Statements. To be more informative 
about what sorts of irrelevant activities the subjects were engaged in the 
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category was subdivided into Interface-Related, Discourse Pacing, and Task 
Definition Irrelevant Statements. 

We anticipate that the nature of the statements occurring in the plans 
will be related to the designs produced by the subjects. We expect that those 
subjects who gave more action, explanative, and problem-constraint 
statements are more likely to be accurate in their solutions. Analyses of the 
course of the design process are expected to reveal the flow of the solution 
activities. This flow will be related to the "planned" flow, with more 
successful designers adapting their plans as necessary. 
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